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ABSTRACT 



The goals, concerns, and issues addressed by a committee of 
the American Association of Teachers of Turkic Languages (AATT) in the 
process of developing a standardized Turkish language proficiency test at the 
college level are examined. The testing committee aimed at incorporating 
recent scholarship on second language acquisition, teaching methods, and 
assessment into development of a test for intermediate and advanced 
proficiency levels. Scoring was designed to indicate both a competitive 
(rating) score and a proficiency level, to meet both diagnostic and placement 
needs. The test incorporates visual and authentic materials. Considerations 
in test administration and in test reliability and validity are discussed 
briefly. (MSE) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 






BEYOND CLASSROOM ACHIEVEMENT: 
STANDARDIZED TURKISH TESTS 

Pel in Ba§ci 

Portland State University 



PERMISSION TO REPRODUCEAND 

disseminate this material has 
beengranted by 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

El This document has been reproduced as 
'’received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



° Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 




2 



Spring 1999 • Fall 1999 



AATT Bulletin 



14 



BEYOND CLASSROOM 
ACHIEVEMENT: 
STANDARDIZED 
TURKISH TESTS 



Pelin Ba§ci 

Portland State University 

I 

An increasing separation of tasks has taken 
place in many “commonly taught” second 
or foreign languages such as English and 
Spanish, where the test builder is no longer 
necessarily the instructor. This is, of course, 
especially true for standardized tests, 
although the same principle also applies to 
oral proficiency assessment. 1 As a matter 
of fact, for some of these language areas, 
assessment is both an independent field of 
research and a full “fledged industry. There 
are advantages and disadvantages to such 
expansions in any field: the advan tag e 
stems from the professional work committed 
to assessment research and implementation, 
while the disadvantage most remarkably 
arises from the commercial interests built 
around highly standardized tests such as the 
Test of English as a Foreign Language 
(TOEFL). In some of these cases, special 
crash courses, study books, and exercise 
tapes all come to function as parts of a vast 
enterprise. 

One doubts seriously that standardized 
tests in Turkish will ever become commer- 
cialized to the extent of, for example, 

TOEFL. While this may indeed be the 
good news, the bad news is that testing 
and assessment of Turkish within the US 
context has hardly reached the widespread 
professionalism that underlies the assess- 
ment research, test preparation, and imple- 
mentation in some of the commonly taught 
language areas. No doubt, this is also due, 
at least in part, to the status of Turkish as ’ 
a non-cognate critical language in a largely 
English-speaking world: there is no com- 
parison between the number of students 
who take Turkish and those who take. 



for example, Spanish or other Romance 
languages in the US. Both the demand for 
standardized tests and the corpus of any data 
that result from such tests and would inform 
assessment research are smaller. Moreover, 
in many US academic institutions, Turkish * 
instructors are asked to strike a difficult 
balance between their increasingly diver- 
sified academic and teaching responsibilities 
and limited resources. 

One of the main goals of the testing 
committee for Turkish, which was founded 
in the late 1990s, has been to broaden the 
scope of interest and research in the testing 
and assessment of Turkish, and thereby 
promote professional awareness of the topic. 
It is with these broader goals in mind that I 
would like to expand further on the ques- 
tions that the testing committee had to 
address in preparing model intermediate 
and advance tests for the 1998 program 
of the American Research Institute in 
Turkey (ARIT) for language study in 
Bogazi^i University. 

Second language acquisition research, 
particularly the debate that centered around 
the distinction between learning and acqui- 
sition, 2 regardless of its shortcomings par- 
ticularly for non-cognate languages, 3 
provided useful analytic categories for re- 
thinking classroom instruction. Debates 
about the place of comprehensible input 
produced in meaningful authentic contexts, 
the presence or absence of the “monitor” as 
a “grammar police” in learners’ minds, and 
the delicate balance between accuracy and 
communicative competence have been 
informing foreign language methodology 
as well as assessment. 4 The preparation 
of multi-dimensional tests, rather than one- 
dimensional paper-and-pen examinations, 
incorporating authentic material that has 
some appeal and relevance to the test-taker’s 
needs, and designing assessment models that 
are user-friendly and tests that help students 
rather than keeping them in their place, have 
been accompanying debates of instructional 
methodology. 5 Moreover, with the national 
move towards clearly defined proficiency 
goals as the organizing principle in te aching 
“knowing” a second or foreign language 
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has been re-defined as the ability to perform, 
“to do” things, to carry out diverse tasks 
using all four language skills in the authentic 
environment of the target language and 
culture. 

The AATT testing committee aimed 
at incorporating much of the useful recent 
scholarship on acquisition, methods, and 
assessment into the process of developing 
model tests for two different proficiency 
levels. In this process, the Turkish pro- 
ficiency guidelines set the framework for 
developing the model tests. However, as 
the committee was charged with the task 
of creating models for the ARIT test, the 
question of intent had to be addressed: to 
what degree were models, like those 
prepared for ARIT, proficiency assessment 
tests? For the models to be successful, other 
significant variables had to be accounted for. 
Prepared within the framework of pro- 
ficiency guidelines, the models had to 
become a point of departure for reliable 
standardized tests that best suited the needs 
of ARIT and reflected the educational 
context that produced the intermediate and 
advanced students of Turkish who would 
take these tests. For the testing committee, 
the identification and recognition of all 
the variables and different criteria in the 
construction of models were of great 
significance, for they helped to define 
what Bachman calls “the abilities we wish 
to measure and the means of measure- 
ment.” 2 

Tests — while by no means the only 
criterion of evaluation — help ARIT decide 
what student to select for an in-country 
study program and whom to reward with 
a scholarship. In this sense, the models had 
to address simultaneously the “selection,” 
“readiness,” and “entrance” goals. Mean- 
while, there was also a greater need for 
the tests to inform instructors and program 
administrators with increased accuracy 
about the students’ readiness to pursue 
intermediate and advanced study in the 
program, which in turn, required that the 
tests “diagnose” student levels and reduce 
the burden of further “placement” pro- 
cedures. 



In addition to selection, readiness, and 
entrance goals, the tests’ ability to diagnose 
and place students into expected levels of 
proficiency implied that the contexts, func- 
tions, and levels of accuracy to be tested 
should reflect the expectations of the re- 
ceiving program. This brought to the fore 
further issues regarding instructional artic- 
ulation, the implied continuity of levels, and 
expectations among different programs both 
within and without the US context — an issue 
that plays a part in decision-making, but is 
beyond the scope of any testing committee’s 
immediate work. 

Having listed some of the goals, major 
concerns, and issues to be addressed, it 
would be thoroughly unrealistic to claim 
with any level of certainty that all of these 
issues were resolved and that “perfect” 
models for standardized national tests 
were in feet created by the Turkish testing 
committee. Yet the following may shed light 
on how the committee handled some of 
these concerns. 

(1) Addressing the objectives: The 
committee designed the scoring to indicate 
both a competitive score and a proficiency 
level in order to meet the different set of 
goals mentioned above. This meant that the 
tests would indicate both a demonstrated 
level of proficiency — as in intermediate-mid 
or intermediate-high — and a numerical score 
showing a ranking of the student in relation 
to other competitors for selection purposes. 

In designing the questions, the com- 
mittee made a serious effort to address 
different proficiency levels for different 
skills. For example, in the section that tested 
speaking, intermediate students were asked 
to communicate concrete, descriptive infor- 
mation about the self and their immediate 
environment; deliver this information 
through a range of speech conventions 
such as questions and commands, using 
grammatical structures such as “var/yok”, 
and the future, present continuous, past, and 
aorist tenses; and display a certain amount 
of control and appropriateness in their use 
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